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DETAILED ACTION 
Response to Arguments 

1 . Applicant's arguments filed 27 May 2005 have been fully considered but they are 
not persuasive. 

The applicant argues that Kim fails to teach or suggest the recited, "single mask 
including data for a plurality of mask overlays corresponding to a plurality of 
[regions/objects] in one of said plurality of video frames." (see Remarks/Arguments: pg. 
15, II. 1-4), specifically, Kim does not include "data for a plurality of mask overlays 
corresponding to a plurality of [regions/objects]. In one of said plurality of video frames, 
where these "mask overlays" are then actuated by a viewer to cause "switching from 
presenting to said viewer a first of one of said plurality of multiplexed video streams to 
presenting to said viewer a second one of said plurality of multiplexed video streams." 
(see Remarks/Arguments: pg. 15, II. paragraph 3). 

The examiner disagrees; Kim is introduced to teach an alternative method for 
identifying objects within a single frame, specifically a single mask including data for a 
plurality of mask overlays corresponding to a plurality of [regions/objects] in one of said 
plurality of video frames (see Kim: col. 2, II. 54-67). Whereas it is recognized that the 
mask overlays of Kim are not actuated by a viewer to cause switching from presenting 
to said viewer a first of one of said plurality of multiplexed video streams to presenting to 
said viewer a second one of said plurality of multiplexed video streams, such is taught 
by the combination of Kaiser and Freeman. One of ordinary skill in the art recognizes 
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the benefits of the providing the alternative overlay scheme of Kim for efficiently and 
uniquely code plural objects within a frame. 

The applicant argues that the segmentation mask does not replace Kaiser's 
single object HTML table and provides no other alternatives to Kaiser's HTML table that 
. would provide for a "single mask including data for a plurality of mask overlays 
corresponding to a plurality of [regions/objects] in one of said plurality of video frames" 
(see Remarks/Arguments: pg. 16, II. paragraph 1) 

The examiner disagrees; Kim provides an alternative method from Kaiser (HTML 
table for identifying an object) by identifying plural images within a frame using a single 
mask corresponding to one of said plurality of video frame. One of ordinary skill would 
readily recognize different types of masks can be used for the purpose of identifying 
images within a frame. 

Further, the applicant argues that Kaiser, Freeman, and Kim fail to teach or 
suggest a " single mask" that includes " location and shape information of said plurality of 
regions in one of said plurality of video frames" (see Remarks/Arguments: pg. 16, II. 
paragraph 3). 

The examiner disagrees; Kim teaches a single mask that includes plural images, 
wherein "each object can be easily identified by the label of the pixels in an object" (Kim: 
col. 2, II. 66-67), wherein the "the segmentation mask a label identifying the region it 
belongs to" (Kim: col. 2, II. 56-58). In other words, each object(s) can be easily 
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identified by a label, wherein the label identifies the region to which is belongs (e.g. 
location and shape information), wherein the location and shape is identified by the 
pixels identified by the mask. 

For the above reasons, the applicant's arguments are not persuasive. 

Information Disclosure Statement 

2. The information disclosure statement filed 16 August 2005 has not been 
considered because corresponding image is not available at the time of examination. 
Consequently, the information disclosure statement will be reviewed in the next action. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 52-75 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
U.S. Patent 6,615,408 to Kaiser et al. in view of U.S. Patent 5,861,881 to Freeman et al. 
and U.S. Patent 6,028,964 to Kim. 

Regarding claims 52, 55, 58, 64, 67, 70, Kaiser teaches a visual highlight as 
shown in figure 6B, which reads on a mask generator for providing a plurality of 
overlays over a series of frames. Kaiser teaches a receiver for receiving the streams 
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and receiving user input (fig. 1, label 1300). Further, Kaiser teaches placing plural 
placement zones regions in a plurality of frames thereby permitting a plurality of zones 
to be presented simultaneously (col. 6, II. 25-34). 

Kaiser teaches receiving selection input identifying an overlay by selecting an 
action to be performed (col. 9, II. 48-65), but Kaiser is silent on switching from one video 
stream to a second stream. Freeman teaches in response to past user inputs, at trigger 
points inserting another video stream being concurrently broadcasted (col. 13, II. 28-45). 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify Kaiser by switching video streams as taught by Freeman 
in order to increase user interactivity and provide desirable programming to the user. 

Kaiser teaches a mask generator of a single overlay for a frame of video (fig. 6B), 
but Kaiser and Freeman are silent on a single mask including data for a plurality of 
mask overlays corresponding to one video frame. In analogous art, Kim teaches a 
identifying plural images within a frame by using pixels constituting the objects, wherein 
the background is represented as a zero value, and object pixels have one of non-zero 

values 1,2 255, identifying the object (col. 2, II. 54-67), which equates to a single 

mask including data for a plurality of mask overlays corresponding to one video frame. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Kaiser and Freeman by using a single mask 
including data for a plurality of mask overlays corresponding to one video frame as 
taught by Kim in order to efficiently and uniquely code plural objects within a frame. 
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Regarding claims 53, 56, 59, 65, 68, 71 , Kaiser teaches sending input to a 
remote location via a backchannel communications connection, as shown in figure 1 
with the Internet as the data network (col. 5, II. 45-54). 

Regarding claims 54, 57, 60, 66, 69, 72, Kaiser teaches location and shape 
information of the regions (fig. 6B, col. 10, II. 20-41). 

Regarding claims 61 and 73, Kaiser is silent on a plurality of video stream 
identifiers associated with one of the plurality of video streams. Official Notice is taken 
that PIDs are well known in the art. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Kaiser by using 
PIDs in order to identify one of the plurality of program streams thereby enabling the 
receiving device to decode desirable packets of data. Kaiser teaches a visual highlight 
as shown in figure 6B, which reads on a mask overlay. Kaiser teaches placing plural 
placement zones regions in a plurality of frames thereby permitting a plurality of zones 
to be presented simultaneously (col. 6, II. 25-34). Kaiser teaches receiving selection 
input identifying an overlay by selecting an action to be performed (col. 9, II. 48-65). 
The mask has corresponding actions that can be taken (fig. 6C, 6D, label 6400), which 
equates to a first data structure element. Kaiser teaches an action resource retrieval 
step (col. 9, II. 48-65), which reads on a second data structure element, but is silent on 
switching from one video stream to a second stream. Freeman teaches in response to 
past user inputs, at trigger points inserting another video stream being concurrently 
broadcasted (col. 13, II. 28-45). Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Kaiser by switching 
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video streams as taught by Freeman in order to increase user interactivity and provide 
desirable programming to the user. 

Kaiser teaches a mask generator of a single overlay for a frame of video (fig. 6B), 
but Kaiser and Freeman are silent on a single mask including data for a plurality of 
mask overlays corresponding to one video frame. In analogous art, Kim teaches a 
identifying plural images within a frame by using pixels constituting the objects, wherein 
the background is represented as a zero value, and object pixels have one of non-zero 
values 1, 2, ..... 255, identifying the object (col. 2, II. 54-67), which equates to a single 
mask including data for a plurality of mask overlays corresponding to one video frame. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Kaiser and Freeman by using a single mask 
including data for a plurality of mask overlays corresponding to one video frame as 
taught by Kim in order to efficiently and uniquely code plural objects within a frame. 

Regarding claims 62 and 74, Kaiser teaches location and shape information of 
the regions (fig. 6B, col. 10, II. 20-41). 

Regarding claims 63 and 75, Kaiser is silent on an audio and private data stream 
associated with the first one of the said plurality of video streams. Official Notice is 
taken the associating an audio and private data stream with video is well known in the 
art. Therefore, it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to modify Kaiser by associating an audio and private data 
stream with video in order to organize related data thereby enabling the receiver to 
efficiently access different streams. 
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5. Claims 76-80 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
U.S. Patent 6,615,408 to Kaiser et al. in view of U.S. Patent 5,861,881 to Freeman et 
al., U.S. Patent 5,818,438 to Howe et al., and U.S. Patent 6,028,964 to Kim. 

Regarding claim 76, Kaiser teaches a visual highlight as shown in figure 6B, 
which reads on a mask generator for providing a plurality of overlays. Kaiser teaches a 
receiver for receiving the streams and receiving user input (fig. 1 , label 1300). Further, 
Kaiser teaches placing plural placement zones regions in a plurality of frames thereby 
permitting a plurality of zones to be presented simultaneously (col. 6, II. 25-34), however 
Kaiser is silent on providing an indication of additional information in order to generate 
zones in response to a user command. Howe teaches providing an icon as an 
indication that additional content is available and in response to seeing the button, 
receiving a request to retrieve information (col. 4, II. 30-50, col. 29, II. 12-27). Therefore, 
it would have been obvious to one of ordinary skill in the art at the time the invention 
was made to modify Kaiser by providing an indication of content and receiving user 
input to perform an action associated with the icon as taught by Howe in order to 
investigate subject matter more thoroughly and according to his or her own tastes 
(Howe: col. 2, II. 48-50). Further, Kaiser teaches receiving selection input identifying an 
overlay by selecting an action to be performed (col. 9, II. 48-65), but Kaiser is silent on 
switching from one video stream to a second stream. Freeman teaches in response to 
past user inputs, at trigger points inserting another video stream being concurrently 
broadcasted (col. 13, II. 28-45). Therefore, it would have been obvious to one of 
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ordinary skill in the art at the time the invention was made to modify Kaiser by switching 
video streams as taught by Freeman in order to increase user interactivity and provide 
desirable programming to the user. 

Kaiser teaches a mask generator of a single overlay for a frame of video (fig. 6B), 
but Kaiser, Freeman, and Howe are silent on a single mask including data for a plurality 
of mask overlays corresponding to one video frame. In analogous art, Kim teaches a 
identifying plural images within a frame by using pixels constituting the objects, wherein 
the background is represented as a zero value, and object pixels have one of non-zero 
values 1,2,...., 255, identifying the object (col. 2, II. 54-67), which equates to a single 
mask including data for a plurality of mask overlays corresponding to one video frame. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Kaiser, Freeman, and Howe by using a single 
mask including data for a plurality of mask overlays corresponding to one video frame 
as taught by Kim in order to efficiently and uniquely code plural objects within a frame. 

Regarding claim 77, Kaiser teaches an overlay associated with a first timing data, 
wherein the overlay is synchronized with the video frame based on the timing (fig. 2, 6B, 
col. 6, II. 18-33, col. 10, II. 39-41). 

Regarding claim 78, Kaiser is silent on comparing the times of data, sleeping for 
a period equivalent to the difference and awakening for visually identifying the object in 
response to an expiration of the time period. Official Notice is taken that use of a 
countdown timer is well known in the art, as is determining the time between events. 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
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invention was made to modify Kaiser by using a countdown timer and determining the 
time between events in order to set an event in the processor to perform an action at 
the predicted time of the next event, thereby reducing the processing tasks and 
enabling the processor to perform other functions simultaneously. 

Regarding claim 79, Kaiser teaches actions (as shown in figure 6C-6D), which 
read on an annotation data associated with the object (col. 9, II. 52-65), which is 
associated with the trigger zone (col. 9, II. 29-51 ). Kaiser is silent on removing the data 
structure in response to determining that the item is no longer used. Official Notice is 
taken that removing expired information is well known in the art. Therefore, it would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to modify Kaiser by removing expired information in order to remote the data structure 
from memory thereby conserving memory and efficiently managing the memory 
resources. 

Regarding claim 80, Kaiser is silent on encoding the data structure and image 
overlay data into a packet stream as recited. Official Notice is taken that PIDs are well 
known in the art. Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to modify Kaiser by using PIDs in order to 
identify one of the plurality of program streams thereby enabling the receiving device to 
decode desirable packets of data. 

6. Claim 81 is rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. 
Patent 6,615,408 to Kaiser et al., U.S. Patent 5,861 ,881 to Freeman et al., U.S. Patent 
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5,818,438 to Howe et al., and U.S. Patent 6,028,964 to Kim in view of U.S. Patent 
6,282,713 to Kitsukawa et al. 

Regarding claim 81 , Kaiser is teaches on encoding the trigger prior to receipt of 
the video (col. 8, II. 14-23), and receiving data structure and the overlay data after the 
receipt of the video at the receiver. Kitsukawa teaches receiving overlay and coupon 
information prior to the receipt of the video (col. 6, II. 51-60). Therefore, it would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
modify Kaiser by receiving the overlay and coupon information prior to receiving the 
video as taught by Kitsukawa in order to provide potential customers with product 
information and incentives to purchase that exploits the convenience of electronic 
purchasing (col. 1, II. 49-55). 

7. Claims 82-89 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
U.S. Patent 6,615,408 to Kaiser et al., U.S. Patent 5,861,881 to Freeman et al., U.S. 
Patent 5,818,438 to Howe et al., U.S. Patent 6,028,964 to Kim, and U.S. Patent 
6,282,713 to Kitsukawa et al. in view of U.S. Patent 6,233,253 to Settle et al. 

Regarding claim 82, Kaiser is silent on a fixed number of packets allocated to 
each of the image overlay data sets, wherein the fixed number of packets corresponds 
to a number of packets filled by the largest image data set. Settle teaches a method for 
inserting NULL packets into an MPEG bitstream. Settle teaches determining the 
multiplexing ratio of the null packets with respect to the video and audio packets (col. 
1 1 , II. 18-35), thereby assigning a fixed number of packets for the data stream, thereby 
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the fixed number a based on the overall bitrate of the channel (col. 10, II. 6-35). 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify Kaiser by determining the packets to be encoded, 
wherein the number of packets is based on the data to be sent as taught by Settle in 
order to encode at the appropriate multiplexing rate various signals thereby maintaining 
a constant bit-rate signal to the receiver. 

Regarding claim 83, Kaiser is silent on using null packets'to fill packets less than 
the fixed number of packets. Settle teaches using null packets to fill the vacant slots 
(col. 4, II. 3-11). Therefore, it would have been obvious to one of ordinary skill in the art 
at the time the invention was made to modify Kaiser by using null packets to fill the 
vacant packet slots as taught by Settle in order to fill the unused data capacity at the 
output thereby maintaining a constant bit rate signal. 

Regarding claim 84, Kaiser is silent on repacking the data stream. Settle 
teaches replacing the null packets with other data packets (col. 10, II. 51-60). 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify Kaiser by replacing the null packets with data packets as 
taught by Settle in order to efficiently distribute data to the receivers. Settle teaches 
ensuring that the buffers are appropriately filled to prevent buffer overflows and 
underflows (col. 1 , II. 45-48), but Kaiser and Settle are silent on packing backwards in 
time. Official Notice is taken that packing data to be received before it is remotely 
processed is well known in the art. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Kaiser by packing 
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the data in order to be received before it is needed thereby ensuring that the data can 
be processed by the receiver thus making the system more robust and readily providing 
the data to the user. 

Regarding claim 85, Kaiser is silent on receiving a packet stream with the video 
stream. Official Notice is taken that the use of data streams in the video stream is well 
known in the art. Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to modify Kaiser by using data streams in the 
video stream in order to efficiently send data to the user. 

Regarding claim 86, Kaiser teaches receiving the packet stream separately with 
an out-of-band channel (col. 5, II. 46-51). 

Regarding claim 87, Kaiser teaches identifying a first two-dimensional section, 
wherein the region is associated with the object and includes one or more pixels, as 
shown in figure 6B. Kaiser teaches filling the region, moving in a first direction along a 
third dimension (e.g. time) (col. 10, II. 20-41). By moving the highlighted portions with 
the frames, the system of Kaiser determines and fills the different sections of the frames 
with respect to time. Kaiser is silent on returning the original highlighted portion and 
moving backwards to fill the area. Official Notice that preprocessing on filling targets is 
well known in the art. Therefore, it would have been obvious to one of ordinary skill in 
the art at the time the invention was made to modify Kaiser by filling the area before 
display in either direction before playback in order to efficiently playback the video with 
the highlighted regions. 

Regarding claim 88, Kaiser teaches a video frame (col. 10, II. 20-41). 
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Regarding claim 89, Kaiser teaches the overlays moving with respect to time 
(col. 10, II. 20-41). 

Conclusion 

8. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 . 1 36(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Andrew Y. Koenig whose telephone number is (571) 
272-7296. The examiner can normally be reached on M-Th (7:30 - 6:30). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Christopher Grant can be reached on (571 ) 272-7294. The fax phone 
number for the organization where this application or proceeding is assigned is 703- 
872-9306. 



Application/Control Number: 09/747,677 



Page 15 



Art Unit: 2611 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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